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A MESSAGE FROM THE FRIEDMAN FOUNDATION: 



OUR CHALLENGE TO YOU 



Our research adheres to the highest standards of scientific rigor. We 
know that one reason the school choice movement has achieved such 
great success is because the empirical evidence really does show that 
school choice works. More and more people are dropping their oppo- 
sition to school choice as they become familiar with the large body 
of high-quality scientific studies that supports it. Having racked up a 
steady record of success through good science, why would we sabotage 
our credibility with junk science? 

This is our answer to those who say we can't produce credible research 
because we aren't neutral about school choice. Some people think that 
good science can only be produced by researchers who have no opin- 
ions about the things they study. Like robots, these neutral researchers 
are supposed to carry out their analyses without actually thinking or 
caring about the subjects they study. 

But what's the point of doing science in the first place if we're never al- 
lowed to come to any conclusions? Why would we want to stay neutral 
when some policies are solidly proven to work, and others are proven 
to fail? 

That's why it's foolish to dismiss all the studies showing that school 
choice works on grounds that they were conducted by researchers who 
think that school choice works. If we take that approach, we would 
have to dismiss all the studies showing that smoking causes cancer, 
because all of them were conducted by researchers who think that 
smoking causes cancer. We would end up rejecting all science across 
the board. 

The sensible approach is to accept studies that follow sound scientific 
methods, and reject those that don't. Science produces reliable empiri- 
cal information, not because scientists are devoid of opinions and mo- 
tives, but because the rigorous procedural rules of science prevent the 
researchers' opinions and motives from determining their results. If 
research adheres to scientific standards, its results can be relied upon 
no matter who conducted it. If not, then the biases of the researcher 
do become relevant, because lack of scientific rigor opens the door for 
those biases to affect the results. 

So if you're skeptical about our research on school choice, this is our 
challenge to you: prove us wrong. Judge our work by scientific stan- 
dards and see how it measures up. If you can find anything in our work 
that doesn't follow sound empirical methods, by all means say so. We 
welcome any and all scientific critique of our work. But if you can't find 
anything scientifically wrong with it, don't complain that our findings 
can't be true just because we're not neutral. That may make a good 
sound bite, but what lurks behind it is a flat rejection of science. 
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Executive Summary 



This is the first empirical study to examine the effects of Ohio’s EdChoice voucher program. Using publicly available 
data, it measures the program’s effect on academic outcomes in public schools where students are eligible for vouchers. 

The EdChoice program offers vouchers to students who are assigned to chronically underperforming public schools. 
Students can use these vouchers to attend private schools of their choice. One of the purposes of voucher programs such as 
EdChoice is to improve academic outcomes at public schools by allowing students to find the schools that are best suited to them 
and by introducing competitive incentives. However, opponents often claim that voucher programs harm public schools. 

This study finds that the EdChoice program produced academic improvements in voucher-eligible public schools. It 
tracks the year-to-year change in test scores within each school from one grade level to the next grade level (e.g. the difference 
between third-grade scores in 2005-06 and fourth-grade scores in 2006-07). It uses regression analysis to compare the academic 
growth of voucher-eligible schools with that of other Ohio schools, controlling for demographic variables and for the presence of 
charter schools. The analyses were then repeated using only schools in districts designated by the state as “major urban— very 
high poverty”; this second round of analysis compares voucher-eligible schools in very poor urban districts to other schools 
in very poor urban districts, helping reduce the possibility that results may be tainted by a statistical phenomenon known as 
“regression to the mean.” Due to the restrictions of available data, the study is unable to include high schools. 



Key findings include: 

| In 2006-07, its first year of operation, the EdChoice program produced substantial academic improvements in Ohio’s 
most stubbornly underperforming public schools. Positive effects were detected in some grades, and no negative 
effects were detected in any grades. 

| The positive effects were substantial in size, though not revolutionary. If the effects accumulate over time, in three to 
four years the voucher-eligible schools will have improved by one standard deviation (equal to one-sixth of the distance 
between the top-scoring and bottom-scoring schools in Ohio). 

| When the analysis was repeated using only schools located in districts designated by the state as “major urban— very 
high poverty,” the results were virtually unchanged. This suggests that the results of the analysis are not affected by 
regression to the mean. 

| The EdChoice program was more restricted in its first year of operation than it is today. Since previous research 
suggests that the positive impact of vouchers on public schools increases when the programs grow, it is reasonable to 
expect that the program’s current benefits probably exceed those detected in this study. 

| This study adds to a large body of empirical research that consistently finds vouchers improve academic outcomes 
at public schools. Vouchers allow families to choose the right schools to meet their children’s needs and introduce 
competitive incentives for improvement that are lacking in the traditional government-run education system. 
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The Friedman Foundation for Educational Choice, dubbed "the nation's leading voucher advocates" by the Wall 
Street Journal, is a nonprofit organization established in 1996. The origins of the foundation lie in the Friedmans' 
long-standing concern about the serious deficiencies in America's elementary and secondary public schools. The 
best way to improve the quality of education, they believe, is to enable all parents with the freedom to choose 
the schools that their children attend. The Friedman Foundation builds upon this vision, clarifies its meaning to 
the public and amplifies the national call for true education reform through school choice. 
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Founded in 1922, Agudath Israel of America is a broadly-based Orthodox Jewish movement with chapters 
in major communities throughout North America. It sponsors a broad range of projects in the fields of 
religion, education, children's welfare, and social action. With the growth of Orthodox Jewry and its 
constituency in America, Agudath Israel has emerged as the leading advocate for the religious and civil 
rights of observant Jews, with an emphasis on education issues. Agudath Israel has played a leading 
role in advocating for educational options for children in Ohio and around the country. 
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Alliance for School Choice 



The Alliance for School Choice is the nation's vanguard organization for promoting, implementing, and 
enhancing K-12 educational choice. In collaboration with a host of national and state allies, we create 
opportunities for systemic and sustainable educational reform that puts parents in charge. 
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Black Alliance for Educational Options 



The Black Alliance for Educational Options (BAEO) is a national, nonprofit, nonpartisan membership organiza- 
tion whose mission is to actively support parental choice to empower families and increase quality educational 
options for Black children. Staunch in its belief that parental choice must be an integral part of any serious effort 
to reform education in America, BAEO believes parental choice programs, which lead to the creation of quality 
educational options, not only rescue the children who can take advantage of such opportunities but also create 
powerful incentives for all schools, public and private, to improve. 



Buckeye Institute for Public Policy Solutions 

The Buckeye Institute for Public Policy Solutions is Ohio's only free market think tank and the leading 
independent source of research and commentary on education, Medicaid, technology regulation reform 
and government accountability. Over the past 14 years, the Buckeye Institute has provided agenda- 
shaping analysis and research to legislative leaders, opinion leaders and the public. 
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The Children's Scholarship Fund of Greater Cincinnati aims to maximize educational opportunity at 
all income levels by offering tuition assistance for low-income families and promoting a diverse and 
competitive educational environment. Since 1999, CSF Cincinnati has been expanding school choice for 
local area students, helping to give all kids an equal shot at educational opportunities. 



Hispanic Council for Reform and Educational Options 
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Hispanic CREO's mission is to improve educational outcomes for Hispanic children by empowering 
families through parental choice in education. We achieve this by providing parents with free informa- 
tion and resources, which help them to become advocates for their children. 
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Northwest Ohio^^ 
Scholarship Fund 



The Northwest Ohio Scholarship Fund, Inc. awards need-based scholarships to low-income families who 
reside in Lucas, Wood or Fulton County in order that their children can attend a private school for Kin- 
dergarten through eighth grade. Each year a lottery is held in July and if a family is selected they submit 
financial information to determine the amount of their child's scholarship, 25%, 50%, or 75% of the total 
tuition up to $1,250. For more information visit www.nosf.org . 



School Choice Ohio 




The mission of School Choice Ohio (SCO) is to protect and expand children's educational options through 
ensuring choice in quality schools and other places of learning, thereby supporting systemic reform of 
K-12 education across the state. The founders of School Choice Ohio believe that all parents should have 
the Opportunity to choose which school is best for their children, regardless of their level of income. 



Thomas B. Fordham Institute 

The Thomas B. Fordham Institute is a nonprofit organization that conducts research, issues publications, 
and directs action projects in elementary/secondary education reform at the national level and in Ohio, with 
special emphasis on our hometown of Dayton. It is affiliated with the Thomas B. Fordham Foundation. 
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Promising Start: An Empirical Analysis of How EdChoice Vouchers Affect Ohio Public Schools 



Introduction 

Ohio’s Educational Choice Scholarship program, often known simply as EdChoice, offers school vouchers to students in chronically 
underperforming public schools. Students can use these vouchers to attend the private school of their choice. This is the first empirical 
study to examine the program’s effects. It uses publicly available data to compare test-score growth in the failing public schools eligible 
for vouchers with test-score growth in other Ohio schools. 

Voucher Competition and Public Schools 

Perhaps the most important concern about school vouchers is the effect they have on public schools. Many people acknowledge 
that vouchers help the students who use them, but are worried that they will make public schools worse by draining money or by 
“creaming” the best students. 

However, the evidence on the real-world effect of voucher programs shows that this is not the case. No empirical study anywhere 
in the United States has ever found that public schools had worse outcomes when exposed to vouchers. And, as we will see in the next 
section, there is a large body of high-quality empirical evidence showing that vouchers make public schools better, not worse. The fears 
that public schools would be harmed by vouchers have simply failed to materialize. The research consistently has found that when 
students can use school choice to attend any school, public or private, the public schools make bigger academic improvements. 

These findings are counterintuitive to many people, but they are not hard to explain. One reason vouchers improve public schools is 
because they allow parents to find the right particular school for each individual child. Every child is unique and has unique educational 
needs. Another reason is that, as the studies below demonstrate, vouchers do not actually drain money or cream students. 

But probably the most important reason school vouchers improve public schools is because they put parents in charge. Vouchers 
give parents a meaningful way to hold schools accountable for teaching their students. 

Under the current system, it is difficult for families to leave if a school isn’t doing a good job. In other service areas, from grocery 
stores to health care, if a service provider isn’t getting the job done, people can switch to another provider simply by making the decision 
to do so. With schools, however, the only way to change is to move— an extremely cumbersome and disruptive step. 

Thus, to a large degree schools can take students for granted. They lack the positive incentive for better performance that, say, 
hospitals have because they know they must do a good job or lose patients. This is especially true for schools that serve low-income and 
disadvantaged students. These populations are less able to move to find a better school. 

With school vouchers, if a public school is providing adequate services, parents can leave their children there and be no worse 
off. But if a public school is not doing an adequate job, parents can go find a private school that will serve their children better. And 
whichever school a family chooses, parents will have the power to hold that school accountable for teaching their children because they 
have the power to leave if they aren’t being served. 

Where parents are empowered with school choice, schools that don’t adequately teach their students will lose them. This provides 
a competitive incentive for better performance. 

Previous Research 

There is a large body of empirical evidence on the question of whether vouchers improve public schools through choice and 
competition or harm them by draining money and creaming students. Unfortunately, this body of evidence often is not taken into account 
in the voucher debate. 

Numerous fiscal studies have examined whether vouchers and tax-credit scholarships (a similar type of school choice program) 
“drain money” from public schools. This body of research has shown consistently that these programs save money both for state budgets 
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and for local public school districts, even after the fixed costs of public schools (costs that do not go away when students leave a school) 
are taken into account. 1 The largest of these studies found that America’s school choice programs have saved a net total of $22 million 
for state budgets and $422 million for local school districts. 2 

The available evidence also does not indicate that vouchers “cream” the best students. The best analysis of this question compared 
voucher applicants in three cities to a representative sample of the eligible population. That study found the two populations to be 
virtually identical on a variety of demographic and educational indicators. 3 

The acid test, however, is what actually happens to public school outcomes when vouchers are implemented. A large body of 
high-quality empirical research has examined this question, using statistical methods to pinpoint the impact of vouchers on academic 
achievement in public schools. 

On this subject, five studies of Milwaukee’s voucher program have been conducted by four research teams, including researchers 
from Harvard, Stanford, the Federal Reserve Bank, the Manhattan Institute and other institutions. 4 Nine studies of voucher programs 
in Florida have been conducted by five research teams, including researchers from Harvard, Princeton, Cornell, the Federal Reserve 
Bank, the Manhattan Institute, the Urban Institute and other institutions. 5 In addition, two studies have examined the impact of voucher 
programs in Maine, Vermont and Texas. 6 

These studies unanimously found that public schools improve when a voucher system has been implemented. Another study of a 
voucher system in Washington D.C. found that vouchers had no visible impact on public schools. 7 The results of the D.C. study should 
come as no surprise, since the D.C. voucher program makes cash payouts to the public school system to compensate for lost students, 
undermining the healthy competitive incentives vouchers would otherwise provide. And even in D.C., vouchers did not visibly harm 
public schools. 

When this research was still in its early stages, some critics noted that where schools are identified by the state as failing, the 
stigma of the failing grade might produce a positive effect on outcomes in those schools. If a voucher program targets failing schools, 
this “stigma effect” might be mistakenly identified as a voucher effect. However, the body of research that has been built up on this 
question has established strong evidence that the positive effect identified from vouchers is not a misidentified stigma effect. Voucher 
programs in Milwaukee and elsewhere that do not target failing schools (and thus cannot get mixed up with a stigma effect) have 
produced positive effects in eight out of nine studies, and in the ninth study (in Washington D.C.) there are special circumstances 
that adequately explain why the voucher failed to produce a positive effect. Among studies of Florida’s failing-schools voucher, four 
specifically tested whether the effect was attributable to stigma; all four found a voucher effect independent of any stigma effect. It is 
not even clear whether a sigma effect existed; two of the studies found no such effect, one found both a voucher effect and a stigma 
effect, and one was inconclusive about whether there was a stigma effect. 8 

Ohio's EdChoice Voucher Program 

In 2005 Ohio enacted a voucher program for students in failing public schools, the Educational Choice Scholarship program, 
commonly called “EdChoice.” Students are eligible for vouchers if they live in the attendance area of a local public school that is 
chronically designated as underperforming. 9 Students can use these vouchers to attend the private school of their choice. However, 
students in Cleveland, who are eligible for another voucher program in that city, are not eligible for EdChoice vouchers. 

Since its original enactment, the program has been expanded periodically to make a larger number of public schools eligible. 
Originally, a school’s students were eligible for vouchers only if it had been designated in a state of “academic emergency” for three 
consecutive years. In early 2006, as students began to apply for the first available vouchers under the program— to be used in the 2006-07 
school year— this definition was expanded. Students who attended schools designated as being in either a state of “academic watch” 
or of “academic emergency” for three consecutive years were eligible for vouchers in 2006-07. 

Later in 2006, the definition was expanded again, but this expansion did not take effect until the 2007-08 school year. As of that 
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year, students are eligible for vouchers if their schools have been designated in a state of “academic watch” or “academic emergency” 
in any two of the previous three years. 

Previous research suggests that the positive impact of vouchers on public schools may get larger as programs increase in size. 
This question has not been extensively studied, since most studies do not measure the impact of a voucher program over multiple 
years. However, the one recent study that did so found large changes in the size of the program’s impact, which tracked closely with 
changes in the program’s size (measured in terms of the percentage of students who used vouchers). 10 

Thus, the changes to the size of the EdChoice program should be considered when evaluating the program’s impact. This study 
examines the impact in the program’s first year, when it was under tighter restrictions. While we do not yet have the opportunity to 
evaluate the program’s impact in the second year, it is reasonable to expect that the impact (whatever it is) will be larger in magnitude 
after the expansion than it was before it. 

Method 

To conduct this study, we obtained test score outcomes for all Ohio public schools in 2005-06 and 2006-07. Specifically, we obtained 
the number of students tested and the average scale score in each school building on Ohio’s state test, broken down by grade level. We 
also obtained the percentage of students in each school who are white, the percentage of students in each school who are eligible for 
the free and reduced-price lunch program, the percentage of schools in each city that are charter schools, and a list of schools that 
were eligible for EdChoice vouchers in 2006-07. 11 

We used math and reading scores from grades three through eight. 12 Ohio also tests in grade ten, but since it does not test in 
grades nine or eleven we were unable to track these scores from year to year. Thus, our study cannot include high schools. 

Because of privacy restrictions, our test score data do not include outcomes for any case in which a school has fewer than ten 
students tested in a grade level. Thus, these cases are excluded from our data. However, because we weighted the data by number of 
students tested (see below) these cases were unlikely to have had an impact on our findings in any event. 

For each school building, we calculated the difference in average scale score in a subject (math or reading) from one grade level 
in 2005-06 to the subsequent grade level in 2006-07. For example, we calculated the difference between a school’s average math score in 
grade 3 in 2005-06 and its average math score in grade 4 in 2006-07, between its average math score in grade 4 in 2005-06 and its average 
math score in grade 5 in 2006-07 and so forth. 

Tracking a cohort of students from year to year in this manner is crucial to sound scientific analysis because it removes most of 
the impact of confounding variables such as demographic factors and unobserved characteristics. While tracking a cohort by school 
building does not provide the same statistical quality as tracking individual students, it is still a good scientific method. And since Ohio 
does not publicly release data based on individually tracked outcomes (as, for example, Florida does), this type of analysis is the only 
one possible with publicly available data. 

Since some cities in Ohio have a robust charter school sector, it was important to control for the presence of charter schools. 
There is not an existing scientific standard for how to do this, since previous studies on the impact of voucher programs on public 
schools have not typically had to deal with the presence of numerous charter schools. We used a straightforward method of controlling 
for the percentage of schools in each city that are charter schools. For most of the more than 700 cities in Ohio, the percentage was 
zero or close to zero. However, for some it was quite large (e.g. Lorain, 36 percent; Dayton, 31 percent; Cleveland, 28 percent; Toledo, 
26 percent). 

We then conducted a linear regression analysis for each test score change variable that we had calculated. In each analysis, the 
test score change variable was the dependent variable. The independent variables were the percentage of students in each school who 
were white, the percentage of students in each school who were eligible for lunch programs, the percentage of schools in the city that 
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were charter schools, and a dummy variable (0 or 1) for whether the school was eligible for vouchers in 2006-07. 

To ensure compatibility, we conducted separate analyses on each change variable rather than combining them. Ohio does not 
report test score results on a single developmental scale that spans all grades (as, for example, Florida does). 

We weighted the data in each analysis by the number of students tested in 2005-06 in the grade and subject that served as the 
starting point for the appropriate change variable. For example, when conducting a regression on the change between grade 3 math 
in 2005-06 and grade 4 math in 2006-07, we weighted by the number of students tested in grade 3 math in 2005-06. 

The schools eligible for EdChoice vouchers are obviously low-performing schools. If positive effects are identified in low-performing 
schools when compared to the general population of schools, this result may be due to a statistical phenomenon known as “regression 
to the mean.” As schools approach the low end of the performance spectrum, they have more statistical “room,” so to speak, to move 
upward rather than downward. This may produce the artificial illusion of a positive effect. 

To test our results for the presence of regression to the mean, we conducted the analyses again using only schools located in 
districts designated by the state as “major urban— very high poverty.” That is, we excluded both voucher-eligible and non-voucher-eligible 
schools that were not located in such districts (since most voucher-eligible schools were located in such districts, this did not have much 
effect on the number of voucher-eligible schools in our data set). If positive results for voucher-eligible schools occur even when we 
confine the data to very-high-poverty urban districts, this would suggest that such results are not tainted by regression to the mean. 

There are a total of 15 school districts designated by the state as “major urban— very high poverty.” However, we excluded the 
Cleveland school district from these analyses because that city has its own voucher program and we need to make sure any public- 
school impact from the Cleveland voucher program does not impact our analysis of the public-school impact of the EdChoice voucher 
program. 13 Thus, our second round of analysis included schools from a total of 14 school districts. 

Results 

The results of our main analyses are summarized in Table 1. They indicate that the EdChoice program had a positive effect on 
academic outcomes in public schools that were eligible. Specifically, a positive effect was identified in some grades, and no negative 
effects were identified in other grades. 

The results of our follow-up analyses to test for regression to the mean are summarized in Table 2. The results are virtually 
unchanged from the main analyses. This suggests that the results are not tainted by regression to the mean. 

The positive effects identified in this study, while not revolutionary, are substantial in size. This is especially true considering 
that they represent only one year’s worth of growth. For purposes of comparison, the standard deviations of the relevant change in 
test scores for all schools in our data set are listed in Table 3. As the table makes clear, if the benefits of vouchers compound over 
multiple years, schools will have realized a full standard deviation’s worth of improvement in three to four years in math and seven 
years in English. 

A standard deviation represents one-sixth of the entire distance from the lowest-scoring school in the data set to the highest- 
scoring school. Thus, a policy with the potential to improve educational outcomes at a school by about a standard deviation in three to 
four years, or even seven years, must be considered substantial. 

Conclusion 

This study identifies some substantial beneficial effects on academic outcomes in public schools from EdChoice vouchers, and 
no harmful effects. Given that only one year’s worth of growth was available for analysis, and that the program was operating under 
stricter limits in its first year, the presence of substantial benefits is strong testimony to the potential of vouchers for improving the 
quality of public schools. 
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Table 1 





EdChoice Vouchers 


Improve Academic Outcomes in 


Ohio Public Schools 


Subject 


Grade Level 


Size of Voucher Effect 


Significance 


Math 


3-4 


(-1 scale point) 


0.486 




4-5 


+ 5 scale points 


0.007** 




5-6 


(-1 scale point) 


0.498 




6-7 


+ 5 scale points 


0.001*** 




7-8 


(-1 scale point) 


0.527 


Reading 


3-4 


(-1 scale point) 


0.151 




4-5 


(0 scale points) 


0.910 




5-6 


( + 2 scale points) 


0.327 




6-7 


+ 2 scale points 


0.043* 




7-8 


( + 1 scale point) 


0.372 



Note: Results are for the change in schools’ average scale scores in each subject on the Ohio state test. The “Significance” column gives the p-value for each result; t = p <= 0.1; * = p <= 0.05; ** = p < = 0.01; *** = p < = 0.001. Control 
variables were included for the percentage of students in each school who are white and low-income, as well as for the percentage of schools in each city that are charter schools. Results were weighted by number of students tested. 



How Do I Read This Chart? 

Regression analysis is used to isolate the impact of one or more variables on a given outcome. For example, we might use 
regression analysis to measure the impact of variables like height and age on the number of points a basketball player scores per 
game. The results are typically reported in the format used in the chart above - each result has a value for the effect size (technically 
called a “coefficient”) and a value for the statistical certainty (the “significance” or “p-value”). Results are only meaningful where 
the statistical certainty reaches at least a certain minimum level— conventionally set at 95 percent certainty, represented by a p-value 
of 0.05, although results above 90 percent certainty (p-value 0.1) can be reported as moderately certain. Other results, where the 
statistical certainty does not reach this level, are regarded as statistical “noise” and do not represent real effects. Traditionally the 
meaningful results are noted using asterisks. Additionally, in the table above the results that are not statistically certain enough to 
be meaningful are put in parentheses. 

As an illustration, an analysis of basketball scores might be reported like this: 





Size of Effect 


Significance 


Height over 6 feet 


+ 7 points per game 


0.003** 


Age over 30 


-11 points per game 


0.041* 


Hair color brown 


(-8 points per game) 


0.784 



This table shows that: 

I Players over six feet tall score seven more points per game than players under six feet tall (once the influence of height 
is isolated from the influence of other factors in the analysis). 

I Players over age 30 score eleven fewer points per game than players under age 30 (once the influence of age is isolated 
from the influence of other factors in the analysis). 

I Having brown hair has no visible impact on points scored per game. 
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Table 2 



Results Do Not Appear to Be Affected by Regression to the Mean 



Analyses repeated using only schools in "major urban-very high poverty" districts 



Subject 


Grade Level 


Size of Voucher Effect 


Significance 


Math 


3-4 


(-1 scale point) 


0.487 




4-5 


+ 4 scale points 


0.031* 




5-6 


(-5 scale point) 


0.108 




6-7 


+ 5 scale points 


0.001*** 




7-8 


(-1 scale point) 


0.495 


Reading 


3-4 


(0 scale point) 


0.940 




4-5 


( + 1 scale points) 


0.644 




5-6 


( + 1 scale points) 


0.704 




6-7 


+ 3 scale points 


0.014* 




7-8 


(0 scale point) 


0.777 



Note: Results are for the change in schools’ average scale scores in each subject on the Ohio state test. The “Significance” column gives the p-value for each result; f = p <= 0.1; * = p <= 0.05; ** = p < = 0.01; *** = p < = 0.001. Control variables were 
included for the percentage of students in each school who are white and low-income, as well as for the percentage of schools in each city that are charter schools. The Cleveland school district was excluded. Results were weighted by number of students 
tested, in the table above the results that are not statistically certain enough to be meaningful are put in parentheses. 



Table 3 



EdChoice Voucher Effects Are Substantial 


Subject 


Grade Level 


Size of Voucher Effect 


Statewide Standard Deviation in Relevant Grade & Subject 

2005-06 2006-07 


Math 


4-5 


+ 5 scale points 


17 scale points 


18 scale points 




6-7 


+ 5 scale points 


19 scale points 


14 scale points 


Reading 


6-7 


+ 2 scale points 


14 scale points 


14 scale points 
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What Is a Standard Deviation? 

Standard deviations are a way of measuring how much variation there is in a given set of values. For example, there is 
a lot more variation in the amount of money people make per year than there is in the number of children they have, so the 
standard deviation for income will be much larger than the standard deviation for number of children. 

In practice, a standard deviation is about one-sixth of the distance between the lowest and highest points in the data set. 
So, to continue the income example, if the lowest income is $20,000 and the highest income is $80,000, the standard deviation 
is $10,000. (Technically, a standard deviation is equal to one-sixth of the distance between the left and right ends of the bell 
curve. ) 




Standard deviations are a common way to gauge the size of an effect. If a study finds that a certain policy produces an 
improvement of “five points,” is that a large impact or a small impact? It depends on how broad a scale you’re dealing with. 
Five points out of ten is a very large impact. Five points out of a hundred is not so large, but may still be quite substantial. 
Five points out of a thousand is probably trivial. Standard deviations allow us to measure how broad the scale is by measuring 
how far apart the high and low ends of the distribution are. 
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statistical clarity (see Patrick Wolf, et. al., "Evaluation of the D.C. Opportunity Scholarship Program: Impacts After Two Years," U.S. 
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